Integrating Probabilistic Modeling and Representation-Building

Optimization algorithms are adaptive when they sample problem solutions based on knowledge of the overall search space gathered from past sampling. Recently, competent adaptive optimization algorithms have been developed that achieve this adaptability via probabilistic modeling of the search space. However, their effectiveness requires the existence of a compact problem decomposition in terms of prespecified solution parameters. This is problematic when solutions do not have a uniform parameterized structure (open-ended solution spaces), or when a compact decomposition requires quantification over a large number of solution parameters (ambiguous semantics). In this context the goal of representation-building is, by exploiting domain knowledge, to transform solution parameters and introduce new parameters to match the underlying problem semantics, allowing compact problem decompositions to be expressed. In summary, I propose a dissertation in support of the thesis: Adaptive optimization algorithms based on probabilistic modeling may be augmented to solve problems with open-ended solution spaces and ambiguous semantics via representation-building. Primary background material is presented in section 1; section 2 describes and motivates the integration of probabilistic modeling and representation-building, including a proof-of-concept case study in the domain of fixed-length bitstrings, with experimental results. The utility of learning open-ended hierarchical structures (symbolic formulae, computer programs, parse trees, etc.), is outlined in section 3, along with some current approaches – I then propose a new approach based on probabilistic modeling and representation-building. Theoretical and experimental evaluation methodologies for this approach are proposed in section 4, and section 5 concludes.